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ABSTRACT 


An  extensive  series  of  design  control  techniques  for  the 
acquisition  of  "systems"  and  "hardware"  has  been  developed  by 
the  Air  Force  over  the  past  five  years.  The  control  of  com¬ 
puter  program  (software)  designs,  the  verification  and  valid¬ 
ation  of  the  computer  program  products,  and  the  integration 
of  "software"  and  hardware  has  not  been  as  aggressively  in¬ 
vestigated.  This  paper  describes  the  overall  Air  rorce  project 
that  has  been  established  to  evolve  a  series  of  techniques  for 
the  acquisition  and  design  control  of  computer  programs  and 
computer  program  data. 

The  Air  Force  375  System  Management  Concept  proved  to  be 
the  best  model  about  which  to  structure  and  develop  the  tech¬ 
nology  and  management  process  for  computer  program  acquisition 
and  development.  To  date,  certain  standards  and  techniques 
have  been  developed;  namely,  the  concept  of  uniform  specifica¬ 
tions,  design  baseline  management,  design  change  controls, 
specification  maintenance,  and  design  accounting.  Current 
studies  are  in  progress  to  evolve  computer  program  testing 
and  validation  standards,  the  systems  engineering  integration 
aspects,  and  the  detailing  of  the  generalized  procedures  and 
events  as  related  to  computer-oriented  systems  and  programs. 
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IN'P'^b'JOTT  ON 


The  classic  approach  to  tre  development  of.  Pir  Force  coornt  ’  r  r.  • ! 
computer  programs  har,  been  to  award  separate  contracts  fcr  the  hard¬ 
ware  and  for  the  " software"  aspects  of  an  electronic  system.  "he 
integration  of  the  hardware  and  the  "software”  into  a  homogenous  total 
system  has  not  always  been  a  planned  certainty,  primarily  due  to  -he 
lack  of  definitive  management  procedures  and  contractor  guidance  and 
control  techniques  for  the  area  popularly  known  as  "software.”  It 
was  with  this  history  in  nind  that  a  studv^  w^s  initiated  to  evolve 
a  technique  which  would  permit  the  following: 

a.  Development  contractual  renuirements  standards  to  pro¬ 
vide  positive  control  over  the  contractor’s  effort  in  the  development 
of  computer  programs. 

h.  Development  of  a  series  of  principles  and  procedures  which 
would  be  applied  to  the  conrigurat ion  management  of  computer  programs. 

c.  Development  or  a  series  of  design  standards  and  specifica¬ 
tions  for  use  during  computer  ^rogra-  design  and  development. 

d.  And  most  important  of  all,  to  develop  an  orderly  process 
of  "software”  production  that  would  assure  ^ull  compatibility  and 
timeliness  with  hardware  production. 

The  acquisition  of  computer  programs,  as  discussed  in  this  paper, 
has  oeen  broadly  categorized  to  two  areas: 

a.  Those  oarlv  events,  activities  and  steps  involved  in 
INITIALLY  acquiring  the  computer  hardware  and  the  related  computer 
programs.  Here  is  the  arena  for  activities  and  problems  from  the 
viewpoint  of  the  system  manager,  the  system  designer,  and  the  system 
acquisition  agenev. 

b.  Those  events  and  activities  by  the  military  user  and 
implementor  to  maintain  and  constantly  update  the  computer  programs 
and  the  computer  program  data  AFTER  the  initial  acquisition.  In  this 
instance,  the  activity  is  primarily  with  the  "software”  implementation 
process  from  the  point  of  view  of  the  "software"  implementor’s  problems 
within  his  own  environment. 

This  paper  and  the  series  of  papers  to  follow,  address  themselves 
to  area  "a",  that  scries  of  events  and  activities  which  involve  the 
initial  design  and  acquisition  of  computer  programs  and  the  integra¬ 
tion  of  the  resultant  efforts  into  the  overall  system.  Those  activities 


identified  in  the  second  category  "b",  are  the  user’s  activities  after 
the  system  is  declared  operational  and  turned  over  to  the  user.  The 
"b"  activities  involve  the  updating  and  maintenance  of  existing  opera¬ 
tional  computer  programs  and  the  evolvement  of  a  limited  number  of  new 
and  additional  programs.  Management  procedures  for  the  ”b”  activities 
are  in  the  process  of  evolvement  by  the  Electronic  Systems  Division  for 
the  U.  S.  Air  Force  and  will  be  presented  at  a  later  date. 

The  objective  of  this  paper  is  threefold: 

1,  To  introduce  the  Air  Force  developed  procedures  for  the 
acquisition  and  control  of  command  system  computer  programs  based  on 
the  concept  of  the  computer  program  as  the  end  item  of  a  definable 
process ; 


2.  To  suggest  that  the  developed  process  is  general  enough 
to  cover  a  wide  variety  of  users  outside  the  military  domain;  and 

3.  In  terms  of  this  process  and  the  Air  Force  system  manage¬ 
ment  philosophy,  set  the  stage  for  the  papers  2»3»**  that  follow. 
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SECTION  II 


THE  COMPUTER  PROGRAM  AS  AM  "END  ITEM" 


A.  Classical  Definition  of  "Software" 


The  term  "software"  is  being  removed  from  official  Air  Force 
terminology  because  of  the  diversity  of  its  entrenched  definitions; 
some  of  the  more  popular  definitions  of  "software"  are  as  follows: 

1.  Computer  programs,  together  with  associated  data, 
training  materials,  and  the  operational  personnel. 

2.  All  computer  programs,  including  the  hardware  of  the 
computer  system  itself. 

3.  All  computer  programs,  excluding  hardware. 

4.  Special  utility  computer  programs  which  are  supplied 
by  the  digital  computer  manufacturer  as  part  of  a  computer. 

5.  Deliverable  contractor  data  associated  with  the  develop¬ 
ment  and  operation  of  system  equipment. 

o 

6.  System  personnel,  as  contrasted  with  system  hardware. 

B.  "Software"  as  an  End  Item 

In  order  to  adapt  the  Air  Force  375  concept  of  system  manage¬ 
ment  to  "software,"  it  was  first  necessary  to  define  "software"  pre¬ 
cisely  and,  more  specifically,  to  attempt  to  describe  it  in  engineering 
and  hardware  terminology.  It  was  determined  that  it  was  more  reasonable 
to  adapt  "software"  to  the  "375  hardware  acquisition  techniques,"  than 
to  develop  a  separate  "software"  management  process,  which  in  turn  would 
require  a  third  or  higher  level  management  process  to  integrate  "soft¬ 
ware"  and  hardware  into  a  total  system. 

Fundamental  to  the  management  of  "hardware"  is  the  readiness 
with  which  it  is  possible  to  identify  an  item  of  hardware  as  an  End 
Item  (a  Contract  End  Item5)  and  the  various  paper  products  (Drawings, 
Manuals,  Specifications)  as  Data  Items.  It  is  this  philosophy  which 
was  adapted  to  "software,"  and  as  depicted  in  Figure  1,  "The  Computer 
Program  as  an  End  Item,"  permitted  the  identification  of  "software" 
products  into  the  more  precise  categories  of : 

1.  Computer  Program  Contract  End  Items  (CPCEI),  and 

2.  Computer  Program  Data 
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HARDWARE 


—  CONTRACT  END  ITEM  (CEI) 


"SOFTWARE" 


—  COMPUTER  PROGRAM  CONTRACT  END  ITEM  (CPCEI) 
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SPECIFICATIONS 
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SPECIFICATIONS 


FIGURE  1 


"THE  COMPUTER  PROGRAM  AS  AN  END  ITEM" 


It  will  be  noted  from  Figure  1  that,  as  is  the  case  with 
hardware,  the  computer  program  is  now  a  designated,  deliverable, 
and  contractually-controlled  product.  The  Data  Items  for  both  hard¬ 
ware  and  for  computer  programs  are  those  paper  products  which  "describe" 
the  deliverable  "end  item"  or  describe  how  to  use,  test,  or  update  the 
end  item.  Card  Decks,  although  physically  "paper1'  are  essentially  in¬ 
tegral  to  the  direct  operation  of  the  computer  itself,  and  are  not  con¬ 
sidered  a  Data  Item  per  se.  Both  the  hardware  and  the  computer  program 
data  items  have  standards  for  their  preparation,  however,  the  format 
and  content  for  computer  programs  is  recognized  in  that  there  are 
separate  "standards"  for  hardware  and  separate  "standards"®  for  com¬ 
puter  program  data  items. 

C.  Definition  of  Contract  End  Item  (CEI) 


The  term  "contract  end  item"  (CEI)  has  been  selected  to 
identify  individual  design  packages.  A  contract  end  item  has  become 
a  very  useful  and  common  reference  for  program  management,  i.e,, 
alignment  of  design  responsibilities,  contracts,  schedules,  budgets, 
etc.  The  CEIs,  thus,  represent  a  computer  program  or  a  level  of 
assembly  of  equipment  selected  as  the  basis  for  management.  Once 
the  "design  package"  has  been  identified  as  a  contract  end  item  (CEI), 
this  end  item  is  then  described  in  an  individual  specification^. 

Experience  to  date  with  electronic  command  systems  contracts^, 
has  indicated  that  there  is  indeed  a  common  basis  for  management  when 
computer  programs  are  designated  as  CEIs;  management  by  the  govern¬ 
ment  of  the  contractor's  efforts  and  management  by  the  contractor  of 
his  own  or  subcontractor's  efforts.  An  understanding  by  both  the 
government  and  the  contractor  of  what  is  to  be  delivered  is  an  im¬ 
portant  step  toward  developing  an  orderly  process  for  the  production 
of  computer  programs. 

D.  Computer  Programs  and  the  375  System  Management  Process 


The  basic  decision  that  a  computer  program  could  be  defined 
as  a  Computer  Program  Contract  End  Item  (CPCEI)  permitted  adaptation 
of  virtually  all  the  Air  Force  375  System  Management  concepts  to  the 
management  of  computer  programs.  Specifically,  the  concepts  of 
uniform  specifications,  baseline  management,  change  control,  speci¬ 
fication  maintenance,  design  reviews,  and  a  test  program  became  all 
as  pertinent  to  computer  programs  as  they  are  to  hardware  acquisi¬ 
tion.  More  importantly,  however,  a  common  definition  and  acquisi¬ 
tion  process  now  assures  a  full  integration  and  in  a  timely  manner 
of  both  the  equipment  and  the  computer  programs  into  an  integrated 
and  homogenous  military  system. 
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There  were  sufficient,  substantive  differences  between 
hardware  end  items  and  computer  program  end  items  to  warrant  the 
development  of  separate  but  companion  procedures  and  standards 
for  computer  programs,  and  these  will  be  described  at  opportune 
moments  during  this  and  the  next  three  (3)  papers.  2*3»4.  However, 
it  was  possible,  in  the  real  live  world  situations  that  the  Elec¬ 
tronic  Systems  Division  is  constantly  exposed,  to  adapt  the  "end 
item"  philosophy  of  management  to  computer  programs,  and  yet  re¬ 
tain  full  technical  integrity  and  full  technical  identity  for  all 
disciplines  concerned. 

E.  Comparison  of  Computer  Program  End  Items  with  Hardware 
End  Items 


Figure  2,  "Some  Commonalities  between  Computer  Program 
.End  Items  and  Hardware  End  Items,"  portrays  in  a  capsule  form 
that  the  major  technical  events  and  milestones,  which  pertain  to 
hardware,  are  also  applicable  directly  to  computer  programs. 

Under  the  Contract  End  Item  (CEI)  concept,  there  is  a  common  and 
understood  method  for  specifying  the  technical  requirements  prior 
to  computer  program  design,  for  specifying  the  interfaces  between 
computer  programs/hardware/personnel,  and  for  the  development  of 
sufficient  computer  program  documentation  to  assure  that  the  user 
can  operate  the  system  effectively. 

The  basic  premise  is  that  it.  is  equally  important  to  specify 
the  system  requirements  for  computer  programs  as  it  is  for  hardware, 
thus  the  initial  common  ground  for  both  is  in  the  System  Specifica¬ 
tion.  In  the  normal  design  process  for  military  or  for  commercial 
computer  programs,  both  the  hardware  designer  and  the  computer  pro¬ 
gram  designer  require  a  document  ilentified  as  a  Specification,  in 
which  the  performance/design  and  test  requirements  and/or  parameters 
can  be  detailed.  Likewise,  design  reviews,  testing,  change  proce¬ 
dures,  manuals,  technical  descriptions  of  the  product  are  equally 
common  to  both  areas.  The  discussion  on  the  system  management 
process  that  follows  attempts  to  point  out  that  it  is  primarily  due 
to  the  "commonality  of  approach"  by  the  computer  program  designer 
and  the  hardware  designer,  that  it  was  possible  to  adapt  the  375 
process  to  computer  program  development. 

Just  as  it  was  possible  to  identify  common  activities,  it 
became  quickly  apparent  that  substantial  differences  existed  too. 
These  are  listed  in  Figure  3,  "Differences  between  Hardware  and 
Computer  Program  End  Items."  Unlike  hardware,  computer  program  in¬ 
structions  do  not  "wear  out."  Further,  it  is  not  necessary  to  build 
and  to  maintain  an  elaborate  special  production  facility  to  produce 
each  computer  program  in  quantity  since,  regardless  of  its  config¬ 
uration,  any  number  of  copies  can  be  normally  duplicated  on  standard 
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FIGURE  2.  "SOME  COMMONALITIES  BETWEEN  COMPUTER  PROGRAM 
END  ITEMS  AND  HARDWARE  END  ITEMS" 


machines  at  small  cost.  Therefore,  the  elimination  of  concepts, 
such  as,  reliability,  spare  parts,  quality  control,  useful  life, 
etc. ,  required  careful  tailoring  of  the  hardware  oriented  manage¬ 
ment  procedures  to  computer  programs.  It  is  felt  that  this  goal 
was  substantially  achieved,  and  the  following  discussion  on  the 
pertinence  and  relative  compatibility  of  the  Air  Force  375  System 
Management  Process  to  computer  program  development  is  an  attempt 
to  demonstrate  that,  with  modified  procedures,  the  basic  375  pro¬ 
cess  can  readily  be  accepted  by  the  computer  program  community. 
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SECTION  III 


CONCEPT  OF  SYSTEMS  MANAGEMENT 


The  contemporary  Air  Force  375  series  regulations  and  the 
AF  Systems  Command  manuals  establish  the  procedures  of  acquiring 
a  new  military  capability  in  a  finite  and  structured  manner. 

These  documents  were  intended  to  include  all  the  activities  and 
all  the  events  necessary  to  assure  a  complete  military  system, 
however,  the  emphasis  in  the  current  AFSC  manuals  is  on  hardware 
acquisition  and  does  not  adequately  identify  the  inherent  pecu¬ 
liarities  of  computer  program  acquisition.  The  "375"  technique, 
nevertheless,  can  be  applied  and  the  computer  program  activities 
readily  associated  with  the  375  life  cycle.  This  is  the  objective 
of  the  discussion  following.  (NOTE:  The  applicability  of  the  AF 
System  Command  Management  Manuals  is  discussed  in  more  detail  in 
the  Appendix  to  this  report). 

The  system  life  cycle  is  divided  into  four  phases:  Conceptual, 
Definition,  Acquisition,  and  Operational,  as  shown  in  Tigure  4. 

A.  An  Overview  of  the  Air  Force  System  Management  Process 


System  management  of  large-scale  military  systems  is  a 
complex  team  effort,  in  which  many  government  agencies  and  the 
contractors  pool  their  resources  to  balance  technology,  cost, 
time  and  other  factors  to  achieve  the  best  performing  product 
for  the  military  inventory.  One  of  the  most  important  points 
to  understand  about  the  375  process  is  that  it  is  not  a  new  way 
of  life,  nor  is  it  an  entirely  new  approach  to  the  development 
of  a  system.  As  stated  by  General  B.  A.  Schriever®,  the  Air 
Force  is  taking  advantage  of  the  lessons  learned  over  past  years, 
and  has  formalized  and  standardized  on  a  "typical"  way  of  doing 
business.  This  is  not  to  say  that  every  program  or  project  was 
or  is  intended  to  follow  all  the  375  procedures  or  all  the  pro¬ 
cesses,  but  that  it  is  a  "road  map"  of  events,  activities,  and 
milestones  that  make  sense,  can  save  the  taxpayers'  dollars,  and 
the  military  time.  The  emphasis  in  the  Air  Force  always  has  been 
on  the  "selective  use"  and  on  the  "discreet  application"  of  these 
formal  and  standard  procedures. 

The  final  portion  of  this  paper  is  essentially  an  intro¬ 
duction  of  a  management  concept  for  computer  programming  and  is 
intended  to  set  the  stage  for  the  papers  2»3»I*  that  follow.  The 
significant  activities  of  the  programming  process,  in  terms  of 
the  four  phases  of  a  system's  life  cycle,  with  an  emphasis  on 
key  milestones  and  key  outputs,  are  summarized  on  a  phase  by 
phase  basis. 
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CONCEPTUAL  PHASE 


Operational  Requirement 
Identified 

Initial  System  Concept 
Evolved 

Design  Requirements  & 
Gross  System  Functions 
Determined 


DEFINITION  PHASE 


System  Specification 
•  Developed 

Functions /Tasks  Defined 
Design  Trade-offs  made 
Specific  Approach  Selected 


ACQUISITION  PHASE 


Detail  Designs  Completed 
Tests  conducted 
Contract  End  Items 
produced 

Logistics /Support  plans 
activated 


OPERATIONAL  PHASE 


System  Tests  completed 
System/Equip,  updated 
System  turned  over  to  User 


FIGURE  4.  AIR  FORCE  SYSTEM  LIFE  CYCLE 


B.  The  Conceptual  Phase  for  a  Computer  Program 


This  is  the  period  during  which  new  inventions,  technolog¬ 
ical  advances,  user  requirements,  time  and  money,  and  other  con¬ 
siderations  are  traded  off  to  define  a  valid  requirement. 

Of  particular  interest  to  computer  program  development  is 
a  firm  delineation  of  the  system  mission,  the  operational  environ¬ 
ment,  interfaces,  and  doctrine  of  operational  employment.  For 
information  management  and  handling  systems,  the  analyses  of  the 
system  information  processing  functions  should  begin  during  the 
Conceptual  Phase.  These  analyses  will  typically  include  the  de¬ 
tailing  of  system  functional  flows,  the  identification  of  design 
requirements,  and  the  performance  of  trade-off  studies  based  on 
estimates  of  cost  effectiveness. 

Figure  5  depicts  tyoical  key  events/activities  that  would 
be  important  to  insure  early  and  adequate  consideration  for  the 
computer  programming  aspects  of  an  information  processing  system. 

Block  1,  "Identify  Operational  Requirement" 

For  computer-based  systems/subsystems,  this  is  the  initia¬ 
tion  of  the  information  processing  analysis.  Mission  requirements 
are  analyzed,  the  operating  concept  is  defined  as  are  the  modes, 
environment  and  constraints.  The  responsibilities,  geographic 
locations  and  the  levels  of  the  user  are  identified.  A  preliminary 
estimate  of  personnel  requirements  is  made.  Depending  on  whether 
the  requirement  is  to  satisfy  a  military  or  a  non-military  user, 
the  information  processing  requirements  are  included  in  the  appro¬ 
priate  planning  documentation,  in  order  to  identify  the  total 
system  operational  requirements. 

Block  2,  "Define  Initial  System  Concept" 

This  step  is  concerned  with  developing  qualitative  require¬ 
ments  of  sufficient  potential  value  as  to  warrant  further  effort 
and  definition  in  specific  terms.  The  qualitative  requirements 
should  recognize  the  status  of  the  pertinent  state-of-the-art  and 
should  conceive  those  concepts  which  will  satisfy  the  basic  mission 
and  plans.  The  initial  operational  functions  and  concepts  are  de¬ 
veloped  at  this  time,  as  would  be  the  identification  and  nature  of 
the  sources  and  the  destinations  of  data  elements.  The  require¬ 
ments  for  data  collection,  generation  and/or  transmission  are 
initially  defined.  It  is  often  pertinent  to  evolve  several  infor¬ 
mation  processing  concepts,  in  order  that  various  approaches  can 
be  assessed,  particularly  from  the  major  time  and  money  trade-off 
viewpoints. 
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FIGURE  5.  THE  CONCEPTUAL  PHASE  FOR  A  COMPUTER  PROGRAM 


Block  3,  "Conduct  System  Feasibility  Studies" 

This  step  represents  the  initial  effort  toward  the  formu¬ 
lation  of  a  specific  operational  requirement  for  a  system.  For 
computer-based  systems,  this  involves  the  translation  of  a  quali¬ 
tative  requirement  into  system  concepts  by  the  assignment  of 
functional  and  operational  capabilities  to  specific  areas  of  equip¬ 
ments,  procedures,  and  man/machine  relationships  on  a  trial  basis. 

This  is  the  first  point  at  which  the  considerations  of  timeliness, 
technological  and  economic  feasibility,  and  capability  are  explored 
in  any  detail. 

Block  4,  "Perform  System  Engineering" 

A  specific  concurrent  and  prime  technical  activity  at  this 
point  in  time  is  the  system  engineering  activity.  In  the  case  of 
military  systems,  this  activity  will  be  performed  more  and  more  by 
in-house  government  agencies  and  less  and  less  by  contractor-industry 
activities.  For  the  Air  Force,  captive  contractors,  such  as  the 
MITRE  Corporation  and  the  Aerospace  Corporation,  and  in-house  engineer¬ 
ing  agencies,  such  as  the  System  Engineering  Group  and  the  Rome  Air 
Development  Center,  perform  the  bulk  of  the  initial  system  engineering 
studies. 


System  engineering  studies  are  conducted  to  the  level  of 
detail  necessary  to  establish  that  the  system  selected  will  meet 
the  operational  performance  capabilities  and  will  be  technically 
attainable  within  the  required  time  period.  Basic  technical  data 
should  essentially  be  complete  in  the  following  areas: 

1.  Information  Processing.  This  will  at  least  include: 
a  description  of  the  mission  requirements,  including  deployment  re¬ 
quirements  and  integration  with  other  systems;  concept  of  operation 
associated  with  each  mission/environment;  the  system  information 
flows;  the  allocation  of  functions  between  man  and  computer;  a  des¬ 
cription  of  the  system  data  base  including  processing  and  storage 
requirements;  a  description  of  selected  techniques  of  computer  pro¬ 
gramming;  a  description  of  the  command  organization  with  preliminary 
estimates  of  personnel  requirements;  a  description  of  requirements 
for  system  exercising;  and  a  description  of  interfaces  with  other 
information  systems. 

2.  Computer  and  Associated  Equipment.  The  performance 
characteristics  of  the  computer,  the  operational  consoles,  the  other 
Input/Output  units,  and  the  special  equipments  for  system  exercising 
are  derived  from  the  analysis  of  system  information  processing  func¬ 
tions,  in  parallel  with  the  determination  of  functions  to  be  performed 
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by  personnel  and  computer  programs.  Both  off-the-shelf  and  state- 
of-the-art  equipments  are  considered.  Additionally,  custom-configured 
or  significantly  modified  existing  equipment  are  included  in  the  cost- 
effectiveness  studies,  in  order  that  technical  feasibility,  cost  and 
schedule  estimates,  and  interfacing  characteristics  with  the  computer 
program  system,  communications,  and  facilities  are  firm  enough  to  war¬ 
rant  the  development  of  a  System  Performance  Specification.  Factors, 
such  as  general,  logical  and  physical  equipment  configuration;  data 
processing  speeds,  storage  requirements,  input-output  interfaces; 
peripheral  requirements  for  magnetic  tape  units  and  card  machines; 
operator  consoles;  special  equipments;  and  growth  potential,  are 
identified  preliminarily  at  this  point  in  time  in  order  to  assure 
feasibility  of  the  system  to  meet  the  mission  requirements. 

Block  5,  ''Determine  Design  Requirements  6  Gross  System  Functions*' 

This  step  signifies  that  sufficient  system  concept  studies, 
system  feasibility  studies,  exploratory  development,  and/or  system 
engineering  have  been  performed  to  satisfy  decision  makers  that  the 
system  costs  are  realistic,  estimated  schedules  realizable,  the  high 
risk  areas  identified,  and  that  this  is  the  system  which  should  be 
funded  and  resources  allocated  with  which  to  proceed  to  the  next 
phases,  the  Definition/Acquisition  Phases. 

C.  The  Definition  Phase  for  a  Computer  Program 

This  phase,  which  came  into  being  under  Secretary  McNamara, 
sees  the  entire  system  defined  and  identified  down  to  the  level  of 
contract  end  item  specifications.  During  this  phase,  the  systems 
analysis  activities  are  carried  out,  considering  such  factors  as 
total  system  cost/schedule/performance,  identification  of  interfaces 
and  high-risk  areas.  The  definition  Phase  culminates  with  the  prep¬ 
aration  of  a  definitized  development  contract  setting  forth  the 
results  of  the  Definition  Phase  as  the  Design  Requirements  Baseline. 

For  electronic  systems,  the  information  processing  aspects 
are  identified  as  a  system  segment.  Technical  studies  and  alter¬ 
native  approaches  are  made  at  this  time  in  the  areas  of  computer 
programming,  equipment,  communications,  facilities,  personnel,  pro¬ 
cedural  data,  and  training,  including  considerations  of  development 
lead  times,  system  testing  criteria.  The  direct  objectives  of  these 
technical  studies  are  to  establish  the  total  spectrum  of  design  re¬ 
quirements  and  design  constraints  firmly,  both  for  hardware  and 
software,  at  the  levels  required  for  the  System  Specification  and 
other  documents  which  must  be  prepared. 

Figure  6  identifies  some  of  the  major  events,  activities, 
and  output  products  that  would  normally  be  expected  to  occur  when¬ 
ever  a  system  goes  through  the  Definition  Phase. 
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THE  DEFINITION  PHASE  FOR  A  COMPUTER  PROGRAM 


Block  6,  "Initial  System  Specification" 

This  step  represents  the  technical  effort  that  is  applied 
to  preparing  a  specific  document  upon  which  many  subsequent  steps 
and  processes  are  dependent.  The  initial  system  specification  is 
largely  oriented  to  the  operational  requirements  and  allows  con¬ 
siderable  latitude  in  the  system  design,  except  in  those  areas 
wherein  gross  design  considerations  are  specified  or  obvious.  In 
some  instances,  expediency  does  not  permit  more  than  a  cursory 
refinement  of  previous  concepts  and  tentative  performance  require¬ 
ments  in  the  formulation  of  this  document. 

The  preparation  of  the  initial  System  Specification  gen¬ 
erally  first  requires  that  trade-off  studies  are  identified  and 
performed  and  that  technical  studies,  consisting  of  reviews,  veri¬ 
fication,  and  expansion  or  alteration  of  technical  concepts  and 
data  resulting  from  the  activities  from  the  Conceptual  Phase  are 
accomplished.  If  time  precludes  in-house  accomplishment  of  these 
studies,  this  activity  may  be  tasked  upon  the  Definition  Phase 
contractors ) .  In  this  case,  the  Definition  Phase  contractor s) 
would  be  required  to  evolve  a  fully  defined  System  Performance 
Specification  as  one  of  his  end  products.  For  computer-based 
systems,  information  processing  functions  are  re-examined  and 
verified  in  relation  to  the  user's  organization,  mission,  and 
operating  doctrines.  Alternative  approaches  in  the  areas  of  com¬ 
puter  programming,  equipment,  communications,  facilities,  person¬ 
nel,  procedural  data,  and  training  are  assessed.  The  direct  ob¬ 
jective  of  these  studies  is  to  establish  the  total  spectrum  of 
design  requirements  and  design  constraints  at  the  levels  required 
for  a  System  Specification. 

Block  7,  "Document  System  Requirements" 

Under  the  375  system  management  concept,  the  entire  tech¬ 
nical  control  of  the  system  is  based  on  the  establishment  of 
three  (3)  baselines/specifications.  The  first  of  these  is  the 
Program  Requirements  Baseline,  the  technical  requirements  of  which 
are  defined  in  the  System  Specification  described  earlier.  The 
Program  Requirements  Baseline  consists  of  documents  which  provide 
a  conceptual  technical  description  of  the  system,  a  description  of 
the  Definition  Phase  inputs/events/expected  outputs  and  the  estima¬ 
ted  cost.  This  is  the  document  that  is  evolved  prior  to  initiation 
of  the  Definition  Phase.  From  the  total  system  approach,  it  is 
critical  that  the  entire  mar./machine  concept  be  sufficiently  de¬ 
tailed  particularly  from  the  time,  money  and  integration  aspects. 


Block  8,  "Expand  System  Requirements" 

The  Definition  Phase  contractors  first  technical  activity 
is  to  conduct  a  comprehensive  and  critical  review  of  the  functions, 
the  performance,  and  the  design  requirements  contained  in  the  System 
Specification.  Objectives  are  to  verify,  expand,  and/or  alter  on 
the  basis  of  the  contractors  approach  and  technical  experience. 
Selected  trade-off  studies  would  be  performed,  interface  require¬ 
ments  of  the  major  hardware  and  computer  program  end  items  would  be 
expanded. 

Block  9,  "Define  Computer  Program  Requirements"  and 
Block  11,  "Define  Equipment  Requirements" 

The  nature  of  the  technical  efforts  will  typically  begin  to 
diverge  at  this  point  in  time  with  respect  to  their  direct  concern 
with  system  operational  functions.  This  effort  will  define  the  re¬ 
quirements  for  the  detailed  tasks  to  be  performed  by  the  operational 
computer  programs  and  the  system  equipment.  It  is  predicated  on  the 
basis  that  the  interfacing  equipment  characteristics  have  been  defined 
at  a  level  which  will  permit  the  analysis  and  specification  of  func¬ 
tions  to  be  performed  by  the  computer  programs  and  equipment  contract 
end  items. 

Block  10,  "Define  Manual  Tasks" 

A  concurrent  activity  is  the  definition  of  manual  tasks. 

The  technical  effort  during  this  phase  encompasses  the  analysis  of 
manual  and  man/machine  tasks.  This  leads  to  the  definition  of: 

1.  Manual  tasks  and  procedures 

2.  Console  operator  procedures  and  decisions 

3.  Design  requirements  for  console  controls  and  displays 
Block  12,  "Part  I  CEI  Specifications" 

One  of  the  most  important  products  of  the  Definition  Phase 
is  the  development  of  all  the  necessary  "design-to"  specifications 
for  the  Computer  Program  Contract  End  Items.  In  Air  Force  terminology 
these  are  called  Part  I  -  Performance  and  Design  Requirements  Specifi¬ 
cations^.  This  part  of  the  CPCEI  specification  is  used  to  specify 
requirements  peculiar  to  the  design,  development,  test  and  qualifica¬ 
tion  of  the  contract  end  items  (CEIs).  The  Part  II  -  Product  Config¬ 
uration  and  Detailed  Technical  Description  Specifications^  are  not 
prepared  until  well  into  the  Acquisition  Phase.  An  important  section 
of  every  computer  program  Part  I  specification  is  the  detailing  of 
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the  requirements  imposed  on  the  design  of  a  computer  program  and 
because  of  its  relationship  to  other  equipment/computer  programs. 
Interfaces  defined  in  this  section  of  the  specification  shall  in¬ 
clude  at  least  all  relevant  characteristics  of  the  computer,  such 
as  memory  size,  word  size,  access  and  operation  times,  interrupt 
capabilities,  and  special  hardware  capabilities.  The  Part  I  speci¬ 
fication  permits  the  first  opportunity  for  initiating  configuration 
management  procedures'1  for  the  control  of  design  requirements  changes. 
Part  I  specifications  permit  competition  for  the  Acquisition  Phase 
contract(s) . 

D.  The  Acquisition  Phase  for  a  Computer  Program 

The  contractor  designs,  produces  and  tests  the  system  hardware 
and  computer  programs  during  this  phase.  This  phase  continues  until 
the  last  article  is  delivered  to  the  operational  user  and  all  system 
development  test  and  evaluation  has  been  accomplished, 

Tigure  7  identifies  many  of  the  important  activities  in  the  design 
and  validation  process  that  would  normally  be  initiated  during  an  Ac¬ 
quisition  Phase. 

Block  13,  "Part  I  Specification" 

Now  that  the  initial  end  item  performance  specifications  have 
been  developed,  the  test  requirements  detailed,  and  all  the  necessary 
man-machine  trade-offs  made,  it  is  necessary  that  the  System  Specifi¬ 
cation  be  updated  and  approved  for  contractual  use  in  the  Acquisition 
Phase  contract(s).  For  hardware,  it  is  usually  possible  to  have  de¬ 
fined  in  detail  the  functional  interfaces,  however,  for  computer  pro¬ 
grams  this  is  normally  not  possible  unless  a  firm  selection  and  approval 
of  specific  hardware,  computer  programs,  and  communications  has  been 
made.  Thus,  the  system  specification  will  normally  not  be  fully  up¬ 
dated,  insofar  as  computer  program  requirements  are  concerned  at  this 
point  in  time,  but  rather,  after  some  of  the  Design  Review  activities 
in  the  Acquisition  Phase. 

Blocks  9,  10,  11  and  12  constitute  the  technical  substance  of 
Design  Requirements  Baseline.  This  baseline  represents  the  technical 
requirements  for  the  Acquisition  Phase  contractor(s) . 

Block  14,  "Preliminary  Computer  Program  Design" 

The  fundamental  purpose  of  the  Acquisition  Phase  is  to  acquire 
and  test  the  system  elements  required  to  satisfy  the  user’s  require¬ 
ments.  Upon  award  of  the  Acquisition  Phase  contract(s),  the  design 
of  all  contract  end  items  begins.  The  Part  I  (design-to)  specifica¬ 
tions  developed  earlier,  plus  the  System  Specification,  will  be  the 
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FIGURE  7.  THE  ACQUISITION  PHASE  FOR  A  COMPUTER  PROGRAM 


contractors  technical  baseline  and,  essentially,  are  a  start  on  the 
design  of  the  end  items.  For  computer  program  end  items,  this  acti¬ 
vity  consists  of  finalizing  the  analysis  of  system  inputs/outputs/ 
functions,  allocation  and  grouping  of  computer  programming  tasks  into 
individual  computer  programs  or  program  packages.  Additionally,  in¬ 
dividual  programs  Will  be  described  and  the  inter-program  interface 
requirements  detailed.  All  this,  plus  the  development  of  functional 
flows,  the  accomplishment  of  trade-offs,  etc.,  will  be  the  basis  of 
the  detailed  design  that  will  be  described  in  the  Part  II  specification. 


Block  15,  "Preliminary  Design  Reviews" 

Block  18,  "Critical  Design  Reviews"  and 

Block  22,  "First  Article  Configuration  Inspection" 


During  the  Acquisition  Phase,  a  series  of  technical  reviews 
and  inspections  are  held  to  provide  discrete  milestones  for  an  ex¬ 
change  of  technical  information  between  the  Air  Force  and  the  con¬ 
tractor.  Two  of  these,  the  Preliminary  Design  Review  and  Critical 
Design  Review,  are  analyses  of  the  design  of  the  contract  end  items 
at  various  stages  of  development.  The  third  milestone,  the  First 
Article  Configuration  Inspection,  is  an  audit  of  the  documentation 
that  describes  the  contract  end  item.  A  more  detailed  discussion  of 
these  milestones  is  contained  in  the  paper  by  Piligian  . 


Block  16,  "Test  Plans"  and 
Block  19,  "Test  Procedures" 

A  parallel  effort  to  the  computer  program  end  item  design  is 
the  development  of  a  Category  I  qualification  test  program.  Draft  of 
the  test  plan  and  associated  test  procedures  for  the  validation  and 
evaluation  of  computer  program  end  items  are  written  for  submission 
to  the  procuring/engineering  agency  for  approval. 

Block  17,  "Detailed  Design  of  Computer  Program  Components  6 
Data  Base" 


The  detailed  design  of  the  computer  program  is  accomplished 
based  on  the  approved  Part  I  Specification  and  the  results  of  the 
Preliminary  Design  Review. 

The  detailed  flowcharts  logic  diagrams,  narrative  description, 
etc. ,  will  provide  the  basis  for  actual  coding  of  the  computer  pro- 
grants).  The  design  of  the  data  base  is  developed  at  this  time  re¬ 
sulting  in  the  number,  type  and  structure  of  tables,  as  well  as 
description  of  the  items  within  the  table.  The  detailed  design 
forms  the  basis  for  the  Critical  Design  Reviews3  discussed  previously. 


Block  20,  "Coding  £  Assembly”  and 

f31ock  21,  "Computer  Propram  Teats" 

Immediately  following  the  Critical  Design  Review,  coding 
the  individual  components  of  the  computer  program,  end  items  takes 
nlace  and  the  process  of  checkout  and  testing  of  the  components 
begins.  Computer  Programs  are  generally  first  tested  during  the 
Category  I3  test  program.  Cat  I  demonstrates  that  the  CPCEI ,  as 
produced,  satisfies  or  does  not  satisfy  the  desipn/nerfornance  re¬ 
quirements  of  the  Part  I  CPCEI  specification. 

Block  23,  "Part  II  Specifications" 

The  Part  II  Specification  is  a  detailed  technical  descrip¬ 
tion  of  the  CPCEI.  Essentially,  it  consists  of  elements  which  have 
appeared  as  computer  program  documentation  in  the  past  in  a  variety 
of  forms  and  combinations— e. g. ,  functional  allocations,  timing  and 
sequencing,  data  base  characteristics  and  organization,  flowcharts, 
etc.  The  Part  II  Specification  organizes  these  into  a  sinrle,  com¬ 
prehensive  description  which  includes  the  data  base  and  listings. 

It  is  an  "as-built"  description,  which  the  procuring  agency  accents 
only  after  its  completeness  and  accuracy  have  been  formally  verified, 
and  normally  after  the  computer  program  performance  has  been  quali¬ 
fied  in  relation  to  the  Part  I  Specification.  Like  the  Part  I,  it 
does  not  contain  operating  instructions,  test  procedures,  or  other 
non-specification  materials.  Its  subsequent  technical  uses  are  as 
an  aid  in  correcting  errors  and  designing  changes.  Following  formal 
verification  of  its  completeness  and  accuracy,  it  defines  the  Product 
Configuration  Baseline. 

Block  24,  "System  Tests" 

The  objective  of  Category  II  system  tests  is  to  demonstrate 
that  the  system  will  satisfy  the  system  performance/design  require¬ 
ments  of  the  System  Specification.  For  computer  programs,  Cat  II 
testing  will  assure  the  overall  systems  engineer  that  the  computer 
programs  are  fully  compatible  with  the  hardware  in  an  integrated 
performance  environment,  and  that  they  meet  the  total  system  require¬ 
ments,  including  personnel,  communications,  man-machine  relation¬ 
ships,  etc. 

E.  The  Operational  Phase  for  a  Computer  Program 

The  Operational  Phase  permits  the  transfer  and  turnover  of  a 
systems/equipments  from  the  design  and  acquisition  agency  to  the 
using  (customer)  agency.  This  phase  begins  when  the  operational 
user  accepts  the  first  article,  when  all  the  elements  of  the  system 
have  been  accepted  by  the  user,  and  ends  when  the  user  disposes  of 
the  last  item  from  the  operational  inventory. 
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Most  Air  Force  systems  are  turned  over  to  the  user  on  an  incre¬ 
mental  basis;  squadron  by  squadron  for  missiles;  site  by  site  for 
electronic  systems;  or  system  element  by  system  element.  In  the  case 
of  computer-oriented  systems  (command/control  systems),  there  is  an 
evolutionary  aspect  present,  which  seldom  permits  the  completion  of 
specific  Operational  Phase,  In  this  instance,  it  might  be  stated  that 
there  may  be  manv  Operational  Phases  for  anv  one  System,  as  a  require¬ 
ment  for  system-updating  occurs  and  as  this  requirement  is  satisfied. 

As  the  development  agency  acquires  end  items,  and  satisfies  the 
user  that  a  group  of  end  items  meets  the  user’s  onerational  require¬ 
ment,  a  formal  release  of  both  the  physical  entities,  as  well  as 
the  engineering  resDonsibility ,  is  made  by  the  development/acquisi¬ 
tion  agency.  For  computer  programs,  the  transfer  of  engineering 
responsibility  would  not  normally  be  to  the  Air  Force  Logistics 
Command  (as  is  true  for  hardware),  but  rather  would  more  likely  be 
to  the  user.  There  are  instances,  however,  where  certain  utility 
and/or  maintenance-diagnostic  computer  programs  furnished  with, 
or  as  parts  of,  the  computer  equipment  are  engineering- transferred 
to  the  Logistics  Command,  rather  than  the  user. 

As  elements  or  end  items  are  turned  over  to  the  user,  the  con¬ 
figuration  management  responsibility  also  becomes  the  responsibility 
of  that  agency.  For  purposes  of  configuration  management  of  computer 
programs,  the  updated  System  Specification  and  the  Part  I  Computer 
Program  End  Item  Detail  Specification  function  as  the  design  require¬ 
ments  baseline  throughout  the  Operational  Phase. 

Figure  8  identifies  many  of  the  specific  activities  that  pertain 
to  a  process  that  permits  the  design  and  acquisition  agency  to  turn 
over  to  the  user  those  completed  elements  of  a  system,  as  the  opera¬ 
tional  requirements  are  met. 

Block  25,  "Turnover  to  User" 

Under  current  Air  Force  concepts,  equipment,  facilities  and 
computer  programs  can  be  incrementally  "turned-over"  to  the  user  on 
an  operating  unit  by  operating  unit  basis.  The  standards  for  computer 
program  turnovers  is  in  the  process  of  evolvement,  however,  it  is  gen¬ 
eral  practice  to  "turn-over"  both  the  hardware  as  well  as  the  applicable 
computer  programs  on  a  concurrent  basis.  Under  any  circumstance, 
turnover  implies  that  the  minimum  operational  capability  has  been 
satisfied  for  the  total  system  or  for  some  element  thereof,  and  that 
the  industry  and  government  design  agencies  consider  the  primary  de¬ 
sign/engineering  activities  as  completed. 
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Block  26,  "Updating  Changes  on  Contract" 

As  with  hardware,  computer  programs  are  subject  to  follow-on 
development  tests  which  normally  result  in  updating  change  require¬ 
ments.  As  the  user  exercises  his  system,  or  sore  element  of  a  svstem, 
deficiencies  and/or  improvements  become  suggested.  Updating  changes 
are  most  practical  to  contracts  which  have  not  been  completed,  and 
can  be  broadly  categorized  as  those  which  are  (1)  the  contractor^ 
responsibility  and  essentially  under  contract  "guarantee,"  and  (2) 
improvements  and  performance  updating  initiated  by  the  user,  which 
requires  a  new  contractual  understanding  and  new  design  effort.  In 
essence,  the  "design-updating"  changes  that  are  necessary  are  now 
placed  on  contract,  system  tested  and  recorded. 

31ock  27,  "Transfer  of  Engineering  f»  Logistic  Pesponsibili  t  ies" 

There  is  no  formalized  Air  Torce  policy  on  the  logistics  and 
engineering  responsibilities  for  computer  programs  once  they  have 
been  turned  over  to  the  user,  however,  an  unofficial  process  is  in 
effect.  Several  large  electronic  systems  have  been  officially 
turned  over  to  using  ccrmands.  The  "engineering"  responsibility 
for  controlling  corrective  actions  and  incremental  improvements  of 
operational  capabilities  through  changes  in  the  computer  programs 
has  in  all  cases,  to  date,  been  assigned  to  and  assumed  by  the  using 
command.  In  some  instances,  this  responsibility  .extends  to  the  main¬ 
tenance  of  handbooks,  user  manuals,  and  other  documents  which  are 
directly  dependent  upon  the  computer  program  end  item  configuration. 
The  "logistics"  and  the  so-called  "engineering"  aspects  of  computer 
programs  are  being  studied  and  debated  at  appropriate  levels,  and 
a  decision  issued,  particularly  with  reference  to  certain  utility 
and/or  maintenance-diagnostic  programs  furnished  with,  or  as  parts 
of,  the  computer  equipment. 

31ock  28,  "User  Operational  System  Tests" 

Upon  the  completion  of  Category  II  system  tests  and  the 
incremental  transfer  of  computer  programs,  computer  equipments, 
handbooks,  user  manuals,  etc.,  to  the  user,  the  using  command  may 
elect  to  conduct  Category  III  type  tests.  Category  III  tests  are 
system  tests  similar  to  Category  II  and  for  computer  programs  may 
either  be  unnecessary,  or  mav  be  conducted  concurrently  with  Cat  II. 

At  this  point  in  time,  the  system  is  fully  operational,  all  live 
environment,  and  is  conducted  bv  the  user,  generally  with  the  assis¬ 
tance  of  contract  services  from  the  automated  data  industry.  As 
during  Category  II  testing,  deficiencies  and/or  improvements  are 
revealed,  and  system  modifications  made  dependent  on  significance 
to  mission  and  budgets. 
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Block  29,  "System  Updating" 

One  of  the  final  technical  functions  is  to  document  and 
record  the  precise  system/equipment  configuration  and  to  update 
all  manuals,  handbooks,  etc.  Beginning  back  in  the  latter  part 
of  the  Definition  Phase,  a  "configuration  management"  procedure 
would  have  been  defined,  agreed  upon  by  the  contractor(s) ,  and 
would  at  this  point  in  time  be  thoroughly  updated  and  turned  over 
to  the  user.  The  user  would  continue  to  use  the  system  specifi¬ 
cation  and  the  Part  I  detailed  "design  to"  specification,  as  the 
design  requirements  baseline  for  the  purposes  of  configuration 
management  and  for  a  design  updating  standard.  The  system  itself 
is  also  updated  to  incorporate  updating  changes  and  modifications 
resulting  from  the  users  operational  testing  and  system  usage. 

Block  30,  "Mission  Performance" 

This  event  represents  the  continued  operation  of  the 
system  by  the  user  until  such  time  as  the  system  is  phased  out 
of  the  operational  inventory. 
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FIGURE  8 


THE  OPERATIONAL  PHASE  FOR  A  COMPUTER  PROGRAM 


SECTION  IV 


SUMMARY  AND  CONCLUSION 


The  management  framework  established  by  the  375  series  Air 
Force  regulations  and  manuals  is  applicable  to  electronic  systems, 
as  well  as  to  aircraft  and  missiles.  This  management  concept  is 
exemplified  by  electronic  computer-based  systems,  such  as  412L, 

416L,  425L  and  465L;  namely,  operational  military  systems  which 
provide  information  processing  assistance  to  command  personnel 
who  are  responsible  for  planning,  decision-making,  and  control 
of  resources  and  forces.  A  major  impact  of  considering  this 
type  of  system  is  to  emphasize  information  processing  and  associ¬ 
ated  technologies  as  the  lead  items  in  system  concept  and  design. 
This  emphasis  is  needed  in  order  to  assure  that  the  specification 
and  procurement  of  hardware  and  facilities  will  be  based  upon 
adequate  consideration  of  the  user’s  information  processing  pro¬ 
cedures  and  requirements. 

The  modification  of  the  current  hardware-oriented  375  system 
management  process  to  include  "software”  activities  places  pri¬ 
mary  emphasis  on  events  associated  with: 

1.  Analysis  of  system  information  processing  functions. 

2.  Allocation  of  functions  to  operational  personnel 
and  the  computer  hardware. 

3.  Definition  of  the  functions  to  be  performed  by  com¬ 
puter  programs. 

4.  Development  of  operating  procedures. 

5.  Design,  development,  and  testing  of  computer  programs. 

6.  Development  of  provisions  for  operational  training 
and  other  personnel  subsystem  products. 

The  implications  of  the  concept  described  in  this  paper  can 
be  summarized  as  follows: 

1.  The  normally  hardware-oriented  System  Program  Director 
is  now  provided  with  a  management  tool  with  which  to  evolve  require¬ 
ments  and  assess  the  total  system’s  progress  by  the  establishment 
of  computer  program  key  events,  milestones  and  activities  during 
the  system’s  life  cycle.  The  "visibility"  aspects  of  computer  pro¬ 
gram  intricacies  will  be  more  evident  to  the  System  managers. 
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2,  Computer  program  requirements  documented  in  a  technical 
specification  provide  a  standard  by  which  the  performance  of  the 
contractor  and  the  performance  of  the  resultant  computer  propram 
can  be  evaluated. 

3.  The  design  documentation  and  specifications  developed 
during  a  normal  Definition  Phase  activity  will  provide  for  the  first 
time  an  opportunity  for  the  government  to  acquire  computer  programs 
on  a  true  price  competition  and  open-bid,  fixed-price  basis,  rather 
than  the  current  predominant  sole-source,  cost-plus  situation. 

The  computer  program  management  techniques  evolved  will 
significantly  reduce  the  excessive  cost-overruns  and  will  reduce 
the  overall  system  costs  that  were  uncontrollable  without  a  finite 
management  control  process. 

5.  The  eventual  user  and  operator  of  the  electronic  system 
will  be  provided  with  sufficient  data  and  documentation  which  will 
permit  the  user  to  operate,  maintain  and  update  the  system  much 
more  effectively  and  efficiently. 

6.  The  concept  described  is  considered  to  be  applicable  to 
any  system,  military,  business,  or  industry  and  to  any  programming 
exercise  that  warrants  the  use  of  computer  program  management  tech¬ 
niques. 
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appi;i:dix 


Tin:  AI?  FORCE  SYSTEM  MANAGE’*! 


MAMMALS 


Tho  Ar  Systems  Command  Manuals  are  the  medium  through  which 
Air  Torce  directives,  regulations,  dictuns  and  requirements  are 
put  into  nracticc.  There  is  a  master  scheme  and  a  specific  usage 
for  each  manual,  in  that  some  are  intended  for  Air  Force  usage 
only  while  others  arc  supplied  select ivelv  as  3  requirement  on 
specific  contracts.  Figure  No.  9  summarizes  those  of  prime  inter¬ 
est  to  industry.  The  manuals  are  cross-re Cerenced  extensively  in 
order  to  eliminate  conflicts  and  double  standards.  The  following 
discussion  describes  the  applicability  of  these  manuals  to  Compu¬ 
ter  Program  acquisition. 

AFSCM  375-3,  System  Program  Office  Manual 

ArSCM  375-3  describes  what  a  System  Program  Cffide  (SPO)  is, 
how  it  is  organized,  how  the  overall  job  is  done,  the  general 
responsibilities,  the  relationships  with  other  government  agencies, 
and  the  functional  duties  and  responsibilities  of  all  the  members 
of  a  SPO.  AFSCM  375-3  is  introductory  and  indoctrinational  in 
nature  and  will  never  be  applied  on  a  contract  as  a  requirement, 
but  is  of  useful  interest  to  contractors.  AFSC^  375-3  requires 
modification  in  order  to  introduce  new  SPO  personnel  to  the  com¬ 
plexities  and  intricacies  of  managing  computer-based  systems. 

AFSCM  375-4,  System  Program  Management  Procedures 

AFSCM  375-4  establishes  the  requirements,  policies,  and  pro¬ 
cedures  for  the  Conceptual,  Definition,  Acquisition,  and  Opera¬ 
tional  Phases  of  a  system  program.  It  prescribes  the  significant 
management  activities  for  integrating  and  fulfilling  the  responsi¬ 
bilities  of  the  organizational  elements  involved  in  managing  a 
total  system  program.  It  is  the  mandatory  management  standard 
for  all  future  Systems  Command  system  programs  and  projects. 

AFSCM  375-4  will  never  be  placed  on  contract  as  a  contractual 
requirement;  however,  it  is  vital  for  contractors  to  be  rsnilinr 
with  375-4,  since  this  is  the  "roadmap"  of  management  and  tech¬ 
nical  milestones  and  events  that  a  SPC  will  follow  on  all  future 
contracts.  ArSCM  375-4  is  the  "overall"  Ar  systems  management 
manual  and  references  375-1  and  375-5. 

As  currently  written,  375-4  does  not  detail  nor  address  itscJc 
to  computer  programs.  The  Electronic  Systems  Division  has,  in 
conjunction  with  the  System  Development  Corporation,  evolved  ESD 
Exhibit  EST-2 ,  which  does  tentatively  describe  computer  program 
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DOD  5010.14  DIRECTIVE 

X 

AFR  375  SERIES  REGULATIONS 

X 

AFSCR  375  SERIES  REGULATIONS 

X 

AFSCM  375-1,  "CONFIG.  MGMT  MANUAL" 

X 

X 

AFSCM  375-3,  "SPO  MANUAL" 

X 

AFSCM  375-4,  "SYS  PROG.  MGMT  PROCEDURES" 

X 

AFSCM  375-5,  "SYSTEM  ENG.  MGMT  MANUAL" 

X 

AFSCM  375-6,  "DEV.  ENG.  MANUAL" 

X 

AFSCM  50-3,  "SYS.  TNG.  EQUIP.  " 

X 

AFSCM  310-1,  "DATA  MANAGEMENT  MANUAL" 

X 

X 

FIGURE  9.  CONTRACTUAL  APPLICATION  OF  "375" 


events  and  milestones.  This  exhibit  is  being  expanded  and  will 
describe  in  detail  all  the  activities,  mi lestor.es  and  events 
relating  to  Computer  Programs,  and  in  particular  relating  the 
hardware  events  with  the  computer  program  events. 


ATS  CM  375-1,  Configuration  Management  Manual 

ArSCM  375-1  establishes  the  poliev,  guidance  and  the  responsi¬ 
bilities  for  system/equipment  in  the  management  of  the  configuration 
of  systens/equipnents.  It  prescribes  the  format  and  the  details  rer 
preparation  and  maintenance  of  specifications  and  draw inns,  it  pro¬ 
vides  for  the  control  and  approval  of  engineering  and  design  changes 
and  ^or  implementing  these  decisions.  It  describes  the  various  engi¬ 
neering  inspections  and  compliance  reviews.  Ar3C*‘  575-1  is  rlao.eu 
on  contracts  as  a  contractual  requirement  on  a  selective  Exhibit  rv 
Exhibit  basis. 


AFSCM  375-1  does  not  currently  provide  for  nrccedures  peculiar 
to  the  area  of  computer  programs.  The  Electronic  Systems  Division 
and  the  System  Development  Corporation  have  developed  an  extensive 
and  detailed  configuration  process  that  complements  the  existin'. 
AFSCM  375-1  process.  Currently,  this  is  available  as  ESD  Exhibit 
EST-1.  The  revised  AFSCM  375-1  will  fully  take  into  account  and 
will  describe  in  detail  the  configuration  management  procedures 
for  computer  programs. 


AFSCM  310-1,  Data  Management  Manual 

Describes  the  overall  data  (drawings,  manuals,  specifications, 
reports,  etc.)  management  approach,  data  control  procedures  and 
data  standards.  AFSCM  310-1  also  is  a  "catalog"  of  approved  data 
(documentation)  items.  Air  Force  organizations  ordering  data  from 
contractors  must  choose  their  documentation  requirements  from  this 
approved  list.  AFSCM  310-1  is  intended  to  be  primarily  a  contrac¬ 
tual  requirement  tvpe  of  manual. 

To  date,  the  following  Data  Items,  suitable  for  contractual 
acquisition  of  computer  program  data,  have  been  developed: 

1.  Positional  Handbook  —  Information  System  Operational 
Personnel 

2.  Contract  End  Iten  Detail  Specification  (Computer 
Program)  Part  I 

3.  Contract  End  Item  Detail  Specification  (Computer 
Program)  Part  II 
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4.  Category  I  Test  Plan  (Computer  Programs) 

% 

5*  Category  I  Test  Procedure  (Computer  Programs) 

6.  Category  I  Test  Report  (Computer  Programs) 

7.  Exercise  Conduct  Manual  (U) 

8.  Synthetic  Inputs  Operator  Guide  (U) 

9.  Evaluation  Manual  (Information  System.  Exercising 
Personnel  (U) 

10.  Human  Operator  Task  Analysis  for  Information  Systems  (IJ) 

11.  Training  Needs/Exercise  Requirements  Analysis  (U) 

12.  Evaluation  Needs/Exercise  Requirements  Analysis  (U) 

13.  Exercising  Capability  Implementation  Plan  (U) 

14.  Minutes  of  Formal  Reviews  and  Inspections 

15.  (Computer  Program)  Users  Manual 

16.  Version  Description  Document  (Computer  Program) 

17*  Change  Status  Report  (Computer  Program) 

18.  Configuration  Index  (Computer  Program) 

19.  System  Exercising  Problem  Package  (U) 

20.  System  Exercising  Problem  Agreements  Documents 
AFSCM  375-5 ,  System  Engineering  Management  Manual 

The  375-5  manual  establishes  and  describes  in  detail  the  Air 
Force  methodology  for  accomplishing  the  system  engineering  manage¬ 
ment  process.  The  main  body  of  the  manual  provides  guidance  and 
policy  for  Air  Force  organizations  and  also  is  very  useful  to 
contractors  for  general  information,  and  an  understanding  of  how 
system  elements  are  laid  out  in  advance  so  that  they  are  under¬ 
stood,  not  only  by  the  contractor  with  respect  to  the  nob,  but 
the  customer  as  well.  It  is  essentially  an  agreement  between 
the  two  parties  who  have  engaged  in  the  contract  as  to  what  they 
expect  to  get,  in  considerable  detail,  at  a  fairly  early  point 
in  time.  This  agreement  —  and  its  controls  —  is  progressively 
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definitized  as  the  system  is  defined.  Then  there  is  a  demonstra¬ 
tion,  as  the  equipment  evolves,  to  show  that  the  contractor  has 
in  fact  achieved  what  was  agreed  upon.  It  will  then  be  possible 
to  look  at  the  dollars,  to  look  at  the  schedule,  and  by  proper 
testing  determine  whether  the  contractor  has  been  successful  in 
producing  the  product  that  was  intended.  With  some  of  the  past 
systems  programs,  this  was  done,  but  not  in  the  same  formalized 
manner.  One  reason,  in  part,  was  that  the  contractor  was  delving 
in  an  area  where  he  was  not  certain  of  the  end  results,  and  the 
customer  was  never  too  certain  of  the  operational  use  of  the  de¬ 
vice  after  he  received  it.  The  customer  frequently  did  not  estab¬ 
lish  the  specific  philosophy  of  operation  and  maintenance  until 
after  the  system  was  developed.  Consequently,  many  changes  occur¬ 
red  in  order  to  satisfy  the  basic  operational  requirements.  In 
the  past  several  years,  both  contractors  and  customers  alike  have 
gained  considerable  experience  and  this  experience  is  now  being 
reflected  in  Air  Torce  regulations  and  manuals. 

AFSCM  375-5  does  recognize  and  identifies  the  basic  relation¬ 
ships  and  interactions  between  computer  programs  and  hardware; 
however,  as  with  all  the  other  AFSC  Manuals  discussed  to  this 
point,  AFSCM  375-5  is  primarily  hardware  oriented.  The  System 
Development  Corporation,  under  a  study  contract  with  the  Elec¬ 
tronic  Systems  Division,  is  currently  developing  in  conjunction 
with  in-house  Air  Force  effort,  a  detailed  exposition  of  the 
system  engineering  process  for  computer  programs.  The  ultimate 
goal  is  to  eventually  develop  one  manual  for  system  engineering, 
to  include  equipment  as  well  as  the  computer  program  processes. 

AFSCM  50-3,  System  Training  Equipment  Management  Manual 

This  manual  is  intended  to  serve  as  a  management  tool  by 
providing  the  procedures,  requirements,  and  responsibilities 
for  the  identification,  design,  development,  documentation, 
acquisition,  test,  and  evaluation  of  all  training  equipment 
required  in  support  of  the  system.  As  currently  in  force, 
the  manual  is  intended  for  both  computer  hardware  and  all 
other  equipment  hardware. 
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13  ABSTRACT 

An  extensive  series  of  design  control  techniques  for  the  acquisition  of 
"systems"  and  "hardware"  has  been  developed  by  the  Air  Force  over  the  past  five 
years.  The  control  of  computer  program  (software)  designs,  the  verification  and 
validation  of  the  computer  program  products,  and  the  integration  of  "software" 
and  hardware  has  not  been  as  aggressively  investigated.  This  paper  describes  the 
overall  Air  Force  project  that  has  been  established  to  evolve  a  series  of  tech¬ 
niques  for  the  acquisition  and  design  control  of  computer  programs  and  computer 
program  data. 


The  Air  Force  375  System  Management  Concept  proved  to  be  the  best  model  about 
which  to  structure  and  develop  the  technology  and  management  process  for  computer 
program  acquisition  and  development.  To  date,  certain  standards  and  techniques 
have  been  developed;  namely,  the  concept  of  uniform  specifications,  design  base¬ 
line  management,  design  change  controls,  specification  maintenance,  and  design 
accounting.  Current  studies  are  in  progress  to  evolve  computer  program  testing 
and  validation  standards,  the  systems  engineering  integration  aspects,  and  the 
detailing  of  the  generalized  procedures  and  events  as  related  to  computer-oriented 
systems  and  programs. 
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